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(54) Method and system for monitoring an interior 



(57) The invention relates to a method and a system 
for monitoring an interior. For detecting an object 3 tres- 
passing into the interior a picture of the interior is com- 
pared with a previously taken background image 
showing the interior from the same angle when it is not 
disturbed by any trespassing object. A detecting means 
30 takes a decision for detecting an object 3 trespassing 
into said interior in the case that there is a shadow 4\ 4 M 
caused by said trespassed object 3 in the picture cover- 



ing at least a section of said interior. In order to reduce 
the effort for detecting the object 3 trespassing into the 
interior the present invention proposes not to evaluate 
shadows in any part of the picture but only that shadows 
which cover at least a part of a predetermined relevant 
area of the interior through which trespassing of an 
object is most likely expected. 
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Description 

[0001] The invention relates to a method for moni- 
toring an interior, in particular the interior of a vehicle, 
and for locating an object trespassing into said interior. 
The method comprises the steps of: detecting an object 
trespassing into said interior in the case that there is a 
shadow caused by the trespassing object in a picture of 
the interior taken when monitoring it, determining the 
location of that section of the interior which has in the 
picture been covered by said shadow, and estimating 
the location of the trespassed object in the interior from 
said determined location of said section in the interior. 
The invention further relates to a system for carrying out 
said method. 

[0002] Such methods and systems are known in the 
art. 

[0003] Fig. 5 for example shows a three dimen- 
sional illustration of the principles known in the art on 
which the present invention is based. More specifically, 
Fig. 5 shows a monitored interior, for example of a vehi- 
cle, into which an object 3 has penetrated. The object 
sticks out through a monitored plain 2 of said interior. 
[0004] In the case that the object 3 has entered the 
interior a shadow 4' caused by said object 3 is recog- 
nised in an infrared or radar picture taken by an optical 
image sensor 1 * for monitoring the interior. Expressed in 
other words, the presence of the object 3 in the interior 
can be detected by the presence of a shadow 4' in said 
picture. 

[0005] The location where said object 3 has pene- 
trated into said interior can roughly be estimated when 
comparing said picture with a background image of said 
interior which has previously been taken by said optical 
image Censor T but riot being disturbed by any pene- 
trating object. More specifically, the location/position of 
the shadow 4* in said picture roughly indicates the loca- 
tion of the penetrating object in the interior. 
[0006] Fig. 6 shows a system known in the art for 
carrying out the above described method. For detecting 
the object 3 penetrating into the interior the system 
comprises said fix -positioned optical image sensor V for 
monitoring said interior. Moreover, it comprises a stor- 
age means 20 for storing the pictures taken by said opti- 
cal image sensor and in particular for storing the 
background image also taken by said optical image sen- 
sor. 

[0007] The system further comprises detecting 
means 30 for comparing the picture with the back- 
ground image and for detecting an object trespassing 
into said interior in the case that there is a shadow in the 
picture covering at least one part of the picture which in 
the case of no disturbance would show the correspond- 
ing part of said background image. 
[0008] A quite similar method and system is for 
example known from DE 41 17 774 A1 . Said document ' 
discloses a method for monitoring an area in order to 
detect and determine target objects in a sequence of 



images taken by a fixed positioned camera and evalu- 
ated by an evaluation circuit. 

[0009] A first image taken by said camera is filtered 
and stored as a start background image in a back- 

5 ground window storage means. The stored start back- 
ground image is subsequently divided into a plurality of 
sectors having equal sizes. Each image subsequently 
taken by said camera is stored in an additional window 
storage means and like said start background image 

10 divided into the same number of sectors having the 
same size. Each sector of a subsequent image is corre- 
lated with the co-ordinated sector of said start back- 
ground image. The results of said correlation process 
are used to roughly calculate the position of said target 

15 objects within said area. 

[0010] Starting from that disclosure it is the object 
of the present invention to improve the method and the 
system for monitoring an interior and for locating an 
object trespassing into said interior in the way that a 

20 decision that an object has in fact trespassed into the 
interior Is ensured. 

[0011] Said object is solved by the subject matters 
of claims 1 and 9, respectively. 

[0012] Starting from the method known in the art 

25 said object is solved by providing the criteria that a deci- 
sion for the detection of the object is only taken, if the 
determined section is located in at least a part of a pre- 
determined relevant area of the interior through which 
trespassing of an object into the interior is most likely 

30 expected. 

[0013] According to the present invention a whole 
interior is monitored in order to recognize undesired 
objects including subjects trespassing into said interior. 
The monitored interior is subdivided into several prede- 

35 termined areas being differently classrfied regarding the 
probability that trespassing of an object into the interior 
is expected. The areas of the interior considered to have 
a high probability that the trespassing of an object is 
expected through them are hereinafter referred to as 

40 relevant areas. 

[0014] The quality of the infrared or radar pictures 
taken for monitoring the interior is often very bad with 
the result that not each shadow shown in the picture 
does in fact represent an undesired object which has 

45 trespassed into the interior. Consequently, a decision 
about the presence of an undesired object in the interior 
taken due to the existence of a shadow in the picture is 
uncertain. 

[0015] In order to overcome that deficiency the 
so decision is - according to the invention - only taken if the 
section of the interior covered by the shadow in the pic- 
ture is located in at least a part of these predetermined 
relevant areas of the interior through which trespassing 
of an object into the interior is most likely expected. In 
55 that way the certainty of the decision is increased. 

[0016] By defining that criteria the decision that an 
object has in fact trespassed into the interior is ensured 
because, to the contrary, trespassing of an object 
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through other less relevant areas of the interior is more 
unlikely or impossible. 

[001 7] According to a first embodiment of the inven- 
tion it is advantageous that the relevant area is a win- 
dow area or at least an area comprising an edge of a 
window since trespassing of an object through a win- 
dow into the interior, in particular of a vehicle is most 
likely expected. To the contrary, trespassing of an object 
through less relevant areas like the ceiling or the floor of 
a vehicle, is less likely. 

[0018] It is of particular interest that the determined 
section allows the conclusion that a part of the trespass- 
ing object has already entered the interior through the 
window area whereas simultaneously another cohesive 
part of the object still remains outside the vehicle. In that 
case it is assumed that parts of the body of a person 
(object), for example the head or an arm has trespassed 
into the interior of the vehicle for example through the 
window, whereas the remaining part of the body of the 
person is still outside of the vehicle. Such a shadow is 
quite different from a shadow of an object which has 
trespassed into the interior as a whole, for example a 
ball which has been thrown into the interior of the vehi- 
cle; such a shadow would only cover inward areas of the 
vehicle. In latter case it is assumed that the trespassing 
object is not a person since there is no part of the 
shadow covering external areas of the vehicle. In that 
way the form and location of the shadow in the picture 
allow conclusions about the trespassing object, in par- 
ticular, as to whether a person or a thing has entered the 
interior. 

[0019] It is advantageous that the determination of 
the location of the section in the interior comprises the 
step of comparing said picture with a background image 
showing the interior from the same angle as the picture 
but not being disturbed by any object which has tres- 
passed into said interior. 

[0020] It is further advantageous that the back- 
ground image can be adapted to changes in the rele- 
vant area of the interior. The background image of the 
relevant section within the interior may be taken when a 
system tor carrying out the method is primarily installed. 
However, in the case, that for example the interior of a 
vehicle is monitored the look of said window area may 
change, for example due to a seat belt which has not 
been coiled up correctly. In that situation it is important 
that said change in the look of the window area is not 
interpreted as a shadow caused by an object which has 
trespassed into the interior. Such a misinterpretation is 
advantageously avoided by providing and using an 
updated background image which has been adapted to 
changes in the relevant section of the interior. Advanta- 
geously the update is done during each restart of the 
alarm system according to the invention. 
[0021] Advantageously an alarm signal is gener- 
ated and forwarded to an alarm station upon the deci- 
sion for the detection of an object within the interior has 
been taken. 
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[0022] Favourably the method is not only able to 
detect an object which has trespassed into the interior 
but also to determine its penetration into the interior. 
[0023] The above defined object of the invention is 
further solved by a system for carrying out the method 
described above. The advantages of said system corre- 
spond to the advantages of said method. 
[0024] The following figures accompany the 
description, wherein 



Fig. 1 shows a 3-dimensional illustration of the 
method for detecting and locating an object 
trespassing into an interior according to the 
present invention; 

Fig. 2 shows side-view of Fig. 1 ; 

Fig. 3 shows a projection of Fig. 1 into a projecting 
plain shown in Fig. 2; 

Fig. 4 shows a system according to the present 
invention. 

Fig. 5 shows a 3-dimensional illustration of the 
principles known in the art on which the 
present invention is based. 

Fig. 6 shows a system for carrying out the princi- 
ples known in the art. 

[0025] In the following a preferred embodiment of 
the invention is described in more detail by referring to 
figures 1 to 4. 

[0026] Fig. 1 illustrates the principle of the present 
invention. 

[0027] In order to monitor an interior, in particular 
the interior of a vehicle there is at least one optical 
image sensor 1 * mounted to the body of the vehicle, for 
example to the ceiling of the vehicle from inside. The 

40 optical image sensor 1 ' takes radar or infrared pictures 
of an area 2 of the interior, preferably upon infrared 
flashing. Since trespassing of an undesired object 3 into 
the interior of the vehicle is a most likely expected 
through relevant areas of the interior like a window area 

45 2 of the vehicle the optical image sensor 1 ' is preferably 
focused to said window area 2. 

[0028] For only generally locating the trespassing 
object one optical image sensor V is sufficient. In the 
case that an undesired object 3 actually trespasses into 

so the interior of the vehicle through said window area 2 it 
causes a shadow 4' in a picture taken by said optical 
image sensor V. Consequently, the existence of a 
shadow in a picture taken by said optical image sensor 
generally allows the conclusion that an object has 

55 entered the interior of the vehicle. Said shadow may in 
particular be recognised when comparing the picture 
with a background image of the same area of the inte- 
rior which has previously also been taken by said optical 
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image sensor 1 ' from the same angle and which is not 
disturbed by any trespassing object. 
[0029] However, in order to more precisely locate 
the penetrating object 3 according to the present inven- 
tion a further optical image sensor 1" is provided for s 
additionally monitoring said window area 2 from a differ- 
ent angle. The penetrating object 3 causes a further 
shadow 4" in a picture taken by said further optical 
image sensor 1" as shown in Fig. 1, irrespective as to 
whether the object 3 just sticks out through a plain io 
spanned by the window area 2 or as to whether the 
object has already completely entered the interior (not 
shown). The more precise location of the penetrating 
object 3 can now be determined by calculating and eval- 
uating the intersection of two fictitious lightening cones is 
5', 5". More specifically, these cones are fictitious trun- 
cated cones wherein the tip of each of said truncated 
cones is located in the centre of one of said optical 
image sensors V, 1" and wherein these cones are 
restricted by the shadow 4\ 4" being co-ordinated to 20 
said optical image sensor and acting as conic section, 
respectively. The location of the object 3 in the moni- 
tored area 2 of the interior is determined by the intersec- 
tion 10 of said shadows 4' and 4". Moreover, penetration 
of the object into the interior is determined by the maxi- 25 
mum distance of the intersection 10 of said truncated 
cones to the plain spanned by said window area 2. 
[0030] The more precise determination of the loca- 
tion of the object and its penetration is now explained in 
more detail by referring to figures 2 and 3. 30 
[0031] For improving the intelligibility of Fig. 3, aux- 
iliary Fig. 2 shows a side-view of the arrangement 
shown in Fig. 1. When the arrangement in Fig. 1 is 
looked at from the line of sight indicated by the arrow 7 
in Fig. 2 one can imagine that it is projected onto a ficti- 35 
tious projecting area 6 shown in Fig. 2. Said projecting 
area 6 is preferably perpendicular to the monitored win- 
dow area / plain 2 of the vehicle and preferably com- 
prises the centres of said optical image sensors V and 

1 ". 40 

[0032] Fig. 3 represents a view onto said fictitious 
projecting area from the indicated line of sight. Conse- 
quently, the 3-dimensional illustration of Fig. 1 is 
reduced to a 2-dimensional illustration which facilitates 
explanation of the determination of the location and the 45 
penetration of the object. 

[0033] Fig. 3 shows a cross-sectional view through 
a car body 8 of the vehicle wherein the external area 9' 
of the vehicle and its interior 9" consequently lie on 
opposite sides of said car body. The car body 8 com- so 
prises the window area 2 through which penetration of 
an object is most likely expected. 
[0034] As already indicated above one optical 
image sensor is sufficient for estimating the location of a 
penetrated object, because its location can be esti- 55 
mated by evaluating the location of its shadow 4', 4" in a 
picture. The location of the shadow is in particular 
defined by its edge 4'a, 4"a, favourably indicated in Fig. 



3 by a fictitious line 5'a, 5"a corresponding to the longest 
section of a line reaching from the centre of an optical 
image sensor V, 1" to the furthermost point of the edge 
4'a, 4 "a of the respective shadow. Said fictitious line 5'a, 
5"a may be considered as being part of the surface area 
of the truncated cones 5', 5" described in Fig. 1 . 
[0035] However, that estimation of the location of 
the object by its shadow is only a rough one when the 
window area 2 is wide in comparison to the width 3a of 
the penetrating object 3. As can be seen from Fig. 3 
there are several possible locations 3, 3', 3" where the 
object 3 could have entered the window area, (see dot- 
ted and solid lines in Fig. 3). In all said different entry 
locations the object 3 would have caused the same fur- 
thermost point of the edge of the shadow. 
[0036] Furthermore, the different possible locations 
3, 3', 3" represent different penetrations P, P\ P" of the 
object 3 into the interior of the vehicle, respectively. 
[0037] In order to more precisely determine the 
location and the penetration of the object into said inte- 
rior the present invention provides a further optical 
image sensor 1 " for monitoring the window area 2 addi- 
tionally from a different angle than sensor 1\ The tres- 
passing object 3 will then additionally cause a further 
shadow 4" on a further picture taken by said further opti- 
cal image sensor 1". When only considering said further 
picture and in particular evaluating fictitious line 5"a 
connecting the centre of said further optical image sen- 
sor 1 " with the far edge 4a of shadow 4'* a similar rough 
estimation about the location and penetration of the 
object 3 would be received as described above. 
[0038] However, the locations of said optical image 
sensors 1 ' and 1 " differ from each other and preferably 
monitor the window area 2 from opposite directions. 
Consequently, the fictitious lines 5 a and 5"a have an 
intersection 10, in particular when they are projected 
into the projecting plain. 

[0039] As can be seen from Fig. 3 calculation and 
evaluation of said intersection 10 allows a more precise 
determination of the location of the trespassing object 3 
(see object 3 drafted in solid lines) with regard to the 
width of said window area. More specifically, in the pro- 
jecting plain there is only one location of the object 3 
(drafted in solid lines) which is co-ordinated to said 
intersection; that specific location can be considered as 
the precise location where the object has actually 
entered the window area 2. Moreover, that specific loca- 
tion allows a more precise determination of the penetra- 
tion P of the object. 

[0040] Fig. 4 shows the system for carrying out the 
method according to the invention. It comprises at least 
the two fix-positioned optical image sensors 1, 1" for 
monitoring the interior from different angles, respec- 
tively. It further comprises a storage means 20 being 
adapted to store at least the two background images 
and / or the pictures taken by said two optical image 
sensors V, 1 ", respectively. The pictures as well as their 
respective background images are provided to a detect- 
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ing means 20 which is able to compare and evaluate at 
least these two pictures and their respective back- 
ground images. Said detecting means 30 takes a deci- 
sion that an object 3 has trespassed into the interior (of 
the vehicle) only if the section of the interior covered in 5 
the picture by the shadow is at least part of a predeter- 
mined relevant area of the interior. Said detecting 
means may further comprise locating means (not 
shown) for estimating the location of the object. 
[0041] The pictures and their respective back- 10 
ground images may be further analysed in a calculating 
means 40 in order to more precisely determine the loca- 
tion and the penetration of said object 3 in the interior. 
More specifically, said calculating means 40 is adapted 
to calculate the penetration of the trespassing object 3 is 
into the interior or the distance between the trespassing 
object and at least of said optical image sensors by 
evaluating the intersection 10 of the fictitious truncated 
cones 5', 5" or of the fictitious lines 5'a, 5 "a mentioned 
above. 20 
[0042] The invention has substantially been 
described by referring to the interior of the vehicles, but 
it shall be pointed out that the principles claimed in the 
present invention are further applicable to other interi- 
ors. 25 



3. Method according to claim 2, characterized in that 
the decision for the detection is only taken, if the 
determined section allows the conclusion that a 
part of the trespassing object (3) has already 
entered the interior through the window area (2) 
whereas simultaneously another cohesive part of 
the object still remains outside the vehicle in front of 
the window area. 

4. Method according to one of the preceding claims 
characterized in that the step of determinating the 
location of the section comprises the step of com- 
paring said picture with a background image show- 
ing the interior from the same angle as the picture 
but not being disturbed by any object (3) which has 
trespassed into said interior. 

5. Method according to claim 4, characterized in that 
said background image is adapted to changes in 
the interior upon each restart of a system for carry- 
ing out the method. 

6. Method according to one of the preceding claims, 
characterized in that an alarm signal is generated 
upon the decision for the detection has been taken. 



Claims 

1. Method for monitoring an interior, in particular the 
interior of a vehicle, and for locating an object (3) 30 
trespassing into said interior, comprising the steps 
of: 



7. Method for determining the penetration of an object 
trespassing into an interior, comprising the steps of: 

carrying out the method according to one of 
claims 1 to 6 at least twice wherein the pictures 
are taken from different angles, respectively; 



detecting an object (3) trespassing into said 
interior in the case that there is a shadow (4 ) 35 
caused by the trespassing object (3) in a pic- 
ture of the interior taken when monitoring it; 

determining the location of that section of the 
interior which has in the picture been covered 40 
by said shadow (4 ); and 



selecting for each of said angles preferably the 
longest section of a line (5'a, 5"a) in the interior 
or in the surface area of a respective fictitious 
truncated cone (5', 5") the tip of which lies in 
the centre of an optical image sensor (1\ 1") 
taking the pictures and which is restricted by 
the area of the shadow (4', 4") in the respective 
picture acting as conic section; 



estimating the location of the trespassed object 
(3) in the interior from said determined location 
of said section in the interior; 45 
characterized in that 

a decision for the detection of the object (3) is 
only taken, if the determined section is located 
in at least a part of a predetermined relevant so 
area (2) of the interior through which trespass- 
ing of an object (3) into the interior is most likely 
expected. 



projecting said selected lines (5'a, 5 "a) into a 
projecting area (6) substantially perpendicular 
to the area of the taken pictures and compris- 
ing at least the centre of one optical image sen- 
sor; and 

calculating the perpendicular distance of the 
intersection 10 of said lines to the area of the 
taken pictures, wherein said distance repre- 
sents the penetration (P) of the object into the 
interior 



2. Method according to claim 1 characterized in that 
the predetermined relevant area (2) is a window 
area or at least an area comprising an edge of a 
window. 



55 8. Method for determining the penetration of an object 
into the interior of a vehicle, comprising the steps 
of: 
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carrying out the method according to one of 
claims 1 to 6 at least twice, however taking the 
pictures from different angles, respectively; 

determining a fictitious truncated cone (5\ 5") s 
for each of said shadows, the tip of said cone 
lying in the centre of the optical image sensor 
(1\ 1") and the cone being restricted by the 
area of the shadow in the picture acting as 
conic section, respectively; 10 

determining the intersection (10) of said ficti- 
tious truncated cones; and 

calculating the maximal perpendicular distance is 
of said intersection (10) to the area of the taken 
pictures, wherein said distance represents the 
penetration of the object into the interior or cal- 
culating the distance from said intersection to 
at least one of the centres of said angles. so 

9. System for monitoring an interior, in particular the 
interior of a vehicle, and for locating an object (3) 
trespassing into the interior, the system comprising: 

25 

a fix-positioned optical image sensor (V) for 
taking a background image and a picture of the 
interior wherein the background image has 
been taken before said picture and is not dis- 
turbed by any trespassing object; and 30 

a storage means (20) for storing at least said 
background image; 

a detecting means (30) for detecting an object 35 
trespassing into said interior in the case that 
there is a shadow (4') in said picture, the 
shadow being caused by said object (3) and 
covering at least a section of said interior in the 
picture; and 40 

a locating means for estimating the location the 
object in the interior by determining the location 
of said section caused by said object in the pic- 
ture; *s 
characterised in that 

said detecting means (30) is adapted to take a 
decision for the detection of the object only if 
the located section is located in at least a part so 
of a predetermined relevant area (2) of the inte- 
rior through which trespassing of an object into 
the interior is most likely expected. 

10. System according to claim 9. characterized in that 55 
the locating means comprises a comparing means 
(40) for comparing said picture with a background 
image showing the interior from the same angle as 



the picture but not being disturbed by any object 
which has trespassed into said interior. 

11. System according to claim 9 or 10, characterized 
in that 

the system comprises at least one further fix- 
positioned optical image sensor (1") for moni- 
toring the interior from a different angle and for 
taking a further background image and a furter 
picture of the interior from said different angle; 

said storage means (20) is adapted to further 
store at least the further background image; 

said locating means are adapted to additionally 
estimating the location of the object by evaluat- 
ing at least one further picture and background 
image of the interior; and 

a calculating means (40) is adapted to calcu- 
late the penetration of the trespassing object 
(3) into the interior or the distance between the 
trespassing object and at least one of said opti- 
cal image sensors (1\ 1") by evaluating the 
intersection of fictitious truncated cones or 
lines within these cones, wherein the tip of 
each of said cones is assumed to lie in the cen- 
tre of one of said optical image sensors 
whereas the bottom of a cone is assumed to be 
represented by the shadow hidden for the 
respective optical image sensor, respectively. 
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